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Glossary 

AHW:  Alberta  Health  and  Wellness 

CCOHTA:  Canadian  Coordination  Office  for  Health  Technology  Assessment 
CHR:  Calgary  Health  Region 
CPG:  Clinical  practice  guideline 

EAP:  Expert  Advisory  Panel  (established  by  Alberta  Health  and  Wellness) 

HTA:  Health  technology  assessment 

HTAI:  Health  Technology  Assessment  Initiative 

HTAU:  Health  Technology  Assessment  Unit 

INAHTA:  International  Network  of  Agencies  for  Health  Technology  Assessment 
IS:  Information  services 

ISTAHC:  International  Society  of  Technology  Assessment  in  Health  Care 

PTCA:  Percutaneous  transluminal  coronary  angioplasty 

RCT:  Randomized  controlled  trial 

RHA:  Regional  Health  Authority 

WHO:  World  Health  Organisation 

WHO  HEN:  World  Health  Organisation  Health  Evidence  Network 
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Introduction 


In  2003,  the  HTAU  at  the  Alberta  Heritage  Foundation  for  Medical  Research  (AHFMR) 
published  a paper  in  its  HTA  Initiatives  series  entitled  'Elements  of  effectiveness  for  health 
technology  assessment  programs' l.  The  paper  was  intended  to  provide  a basis  for 
discussion  on  the  components  of  HTA  programs  and  approaches  to  measuring  their 
effectiveness.  It  gave  some  consideration  to  better  defining  the  role  played  by  HTA 
agencies  and  the  determinants  of  their  effectiveness.  These  determinants  are  outlined  in 
Figure  1.  In  part,  the  paper  was  seen  as  a self  assessment  tool  for  HTA  program 
managers  to  help  achieve  continual  improvement  of  activities. 

Figure  1:  Determinants  of  effectiveness 
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The  HTAU  has  also  been  exploring  ways  to  assess  the  impact  of  its  products  and  has 
published  two  recent  reports  on  this  topic  which  included  responses  to  surveys  of  the 
Unit's  clients  2> 3. 
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This  paper  has  been  prepared  to  provide  an  overview  of  the  outputs  of  the  HTAU  as  an 
aid  to  future  management  of  HTA  activities  at  the  AHFMR,  drawing  on  concepts 
outlined  in  an  earlier  HTA  Initiatives  publication  (HTAI  9) 1.  This  is  the  first  time  that 
this  framework  has  been  applied  to  an  HTAU.  In  addition  to  its  application  to  the 
Alberta  program,  this  paper  forms  part  of  an  effort  to  help  the  international  HTA 
community  come  to  grips  with  how  HTA  programs  may  be  evaluated  on  a rigorous 
and  consistent  basis.  The  paper  is  also  intended  to  complement  the  previous  papers  on 
HTA  impact  as  a further  approach  to  defining  performance  and  accountability  of 
programs  in  a complex  area  that  has  still  received  relatively  little  attention. 

The  paper  reviews  the  activity  of  the  HTAU  as  reflected  in  the  various  products  it 
produced  from  2002  to  2003,  including  the  impact  of  these  on  decision  makers  in  the 
province,  and  identifies  some  areas  for  consideration  in  future  management  of  the 
HTAU. 
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Approach 


The  scope  and  other  details  of  the  project  were  discussed  with  HTAU  staff  in  May  and 
June  2003  and  subsequently  confirmed  with  the  Director  of  the  HTAU.  It  was  agreed 
that  the  overview  would  be  based  on  scrutiny  of  products  completed  from  April  2002  to 
March  2003  inclusive.  All  types  of  product  from  the  Unit  were  considered  down  to 
'Level  D'  QwikNotes  or  their  equivalent  (see  Appendix  A).  Very  brief  and  rapid 
responses,  such  as  "instant  e-mail",  were  not  included. 

An  information  collection  form  was  developed  with  input  from  HTAU  staff  (Appendix 
B).  Unit  staff  then  used  the  form  to  provide  details  of  the  products  for  which  they  had 
been  responsible.  Two  rounds  of  follow-up  requests  for  further  information  were 
undertaken  to  clarify  various  points,  including  summaries  of  the  conclusions  reached  in 
assessments.  Staff  had  the  opportunity  to  correct  and  update  material  entered  on  the 
information  collection  form. 

The  information  was  then  used  to  provide  summaries  of  various  areas  related  to  the 
preparation  and  use  of  the  products.  Most  of  the  details  in  the  paper  relate  to 
Formulation  of  HTA  questions,  HTA  products.  Dissemination,  and  Impact  of  HTA, 
which  were  'Assessment  Stream'  issues  identified  in  HTAI  9.  Some  consideration  is 
also  given  to  Governance,  Resources,  Staff  and  Structure  as  well  as  Collaborative  and 
Contractual  Inputs. 

The  Unit's  products  included  both  assessments  and  resource /educational  publications. 
These  categories  are  discussed  separately  since  the  issues  related  to  them  are  somewhat 
different. 
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HTAU  Products 


The  nineteen  assessment  products  completed  from  April  2002  to  March  2003  are  listed 
by  category  in  Table  1.  Four  of  these  were  in  the  HTA  series.  There  were  also  five 
TechNotes,  two  HTA  Initiatives  reports,  two  Information  Papers,  two  QwikNotes,  and 
four  reviews  that  fell  outside  the  regular  HTAU  series.  The  latter  had  a substantial 
influence  on  the  workload  and  operation  of  the  Unit.  One  of  the  Initiatives  reports  was 
prepared  at  the  University  of  Alberta  and  published  by  the  AHFMR  in  an  effort  to 
further  links  with  the  Department  of  Public  Health  Sciences  and  raise  the  profile  of 
HTA  in  the  province. 

Table  2 lists  the  three  educational /resource  products  completed  during  this  period  - the 
latest  update  of  an  Information  Paper  and  two  publications  in  the  HTA  Initiatives 
series. 

Table  1:  Assessment  products  April  2002  to  March  2003 

Health  Technology  Assessments 

01  Brooks  L,  Leggett  Tait  P,  Harstall  C.  Acupuncture:  evidence  from  systematic  reviews  and  meta- 
analyses, May  2002 

02  Ospina  M,  Harstall  C.  Prevalence  of  chronic  pain:  an  overview,  December  2002 

03  Ospina  M,  Harstall  C.  Multidisciplinary  pain  programs  for  chronic  pain:  evidence  from  systematic 
reviews,  January  2003 

04  Guo  B,  Scott  A,  Bowker  S.  Suicide  prevention  strategies:  evidence  from  systematic  reviews, 
February  2003 

TechNotes 

05  Ospina  M.  Cryosurgery  for  prostate  cancer,  May  2002 

06  Guo  B.  Intracoronary  brachtherapy  for  the  treatment  of  in-stent  restenosis,  May  2002 

07  Guo  B.  Osteogenic  protein-1  for  fracture  healing,  November  2002 

08  Scott  A.  Trigger  point  injections  for  non-malignant  chronic  pain,  March  2003 

09  Scott  A.  Treatment  of  thoracic  insufficiency  syndrome  with  the  vertical  expandable  prosthetic 
titanium  rib,  December  2002 

HTA  Initiatives 

10  Hailey  D.  Minimally  invasive  hip  arthroplasty,  March  2003 

1 1 Yoon  P.  Emergency  department  fast-track  system,  March  2003 
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Table  1:  Assessment  products  April  2002  to  March  2003  (cont’d) 

Information  Papers 

12  Hailey  D.  Stereotactic  radiosurgery:  an  update,  May  2002 

13  Hailey  D.  Hyperbaric  oxygen  therapy  - recent  findings  on  evidence  for  its  effectiveness,  March 
2003 

QwikNotes 

14  Guo  B.  Effectiveness  of  computerized  physician  order  entry  systems  in  reducing  medication 
errors,  April  2002 

15  Corabian  P.  Prolotherapy,  June  2002 

‘Customized  Responses’  - reviews  undertaken  outside  regular  HTAU  series 

16  HTAU . A selected  inventory  of  abstracts  of  Cochrane  reviews  on  physiotherapy/physical  therapy, 
October  2002 

17  HTAU.  A selected  inventory  of  abstracts  of  systematic  reviews  on  chiropractic  services,  October 
2002 

18  HTAU . A selected  inventory  of  abstracts  of  systematic  reviews  on  podiatry  services,  October 
2002 

19  HTAU . A selected  inventory  of  abstracts  of  systematic  reviews  on  optometric  sen/ices,  October 
2002 


Table  2:  Resource  and  educational  products,  April  2002  to  March  2003 

20  Chan  L,  Topfer  LA.  Health  technology  assessment  on  the  Net:  a guide  to  internet  sources  of 
information,  June  2002  (Information  Paper) 

21  Hailey  D.  Local  health  technology  assessment:  a guide  for  health  authorities,  December  2002 
(HTA  Initiative) 

22  Hailey  D.  Elements  of  effectiveness  of  health  technology  assessment  programs,  March  2003 
(HTA  Initiative) 
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Formulation  of  HTA  Questions 


Sources  of  requests  for  assessments  are  shown  in  Table  3.  The  pattern  is  similar  to  those 
for  previous  years,  with  most  products  being  developed  in  response  to  requests  from 
AHW  and/or  RHAs.  All  the  HTA  products  were  aimed  at  decision-makers  within  the 
province.  Two  of  them  were  also  prepared  with  the  aim  of  furthering  HTA  in  Alberta. 

Table  3:  Sources  of  requests  for  HTA  products 


AHW 

RHAsa 

AHW  + 
RHAs 

Other 

HTAb 

Professional 

body0 

University' 

lift  1 ililllll 

HTA  report 

1 

3 

TechNote 

2 

2 

1 

HTA  initiatives 

1 

1 

Information  Paper 

1 

1 

QwikNote 

1 

1 

Customized  Response 

4 

Totals  8 4 

a.  Includes  the  Alberta  Mental  Health  Board 

4 

1 

1 

1 

b.  Calgary  Health  Technology  Implementation  Unit 

c.  College  of  Physicians  and  Surgeons  of  Alberta 

d.  University  of  Alberta,  Department  of  Public  Health  Sciences 

Information  on  the  specification  of  the  assessment  products,  including  details  of  the 
issues  that  were  to  be  addressed,  are  given  in  Table  4.  The  responses  provided  by 
HTAU  staff  indicated  that  project  specifications  were  addressed  mostly  in  terms  of  the 
question  or  issue  raised  by  the  organization  requesting  the  HTA.  In  all  cases  there  was 
discussion  to  confirm  or  clarify  the  nature  of  the  request  (often  face  to  face,  also  by 
telephone  and  e-mail).  Most  of  the  questions  were  framed  in  terms  of  evidence  of 
effectiveness  or  current  status  of  various  technologies,  sometimes  in  association  with 
safety  issues.  Cost  or  access  issues  were  mentioned  less  often.  This  is  similar  to  the 
pattern  for  other  HTAU  products  in  previous  years. 

In  several  cases,  the  product  was  a follow  up  to  a previous  assessment  undertaken  by 
the  HTAU  or  was  intended  as  a precursor  to  further  work  on  a topic. 
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Table  4:  Specification  of  HTA  products 


Product 


Question/issue 


Health  Technology  Assessments 


01  Acupuncture: 
evidence  from 
systematic  reviews 
and  meta-analyses 


Current  scientific  evidence  over  the  last  five 
years  addressing  the  question  ‘for  which 
conditions  acupuncture  has  scientific  support’ 
(effectiveness,  for  whom,  by  whom) 


02  Prevalence  of  chronic 
pain:  an  overview 


What  is  the  best  available  evidence  on 
prevalence  estimates  for  pain  not  related  to 
cancer? 


03  Multidisciplinary  pain 
programs  for  chronic 
pain:  evidence  from 
systematic  reviews 


Evidence  on  the  efficacy/effectiveness/ 
efficiency  of  multidisciplinary  pain  programs 
for  non-malignant  chronic  pain  and  how  do  we 
establish  provincial  needs  for  a multidiscipline 
pain  program? 


04  Suicide  prevention  What  suicide  prevention  strategy  is  effective? 
strategies:  evidence 
from  systematic 
reviews 


05  Cryosurgery  for 
prostate  cancer 

06  Intracoronary 
brachytherapy 


TechNotes 

What  is  the  status  of  cryosurgery  for  the 
treatment  of  prostate  cancer? 

The  current  status  of  the  use  of  intracoronary 
brachytherapy  for  patients  with  in-stent 
restenosis  after  PTCA 


07  Osteogenic  protein-1  Regulatory  status  in  Canada  and  the  US  for 
for  fracture  healing  patient  groups;  current  practice  in  Canada; 

and  evidence  supporting  its  use  and  on 
adverse  effects 


08  T rigger  point  Is  trigger  point  injection  effective  for  treating 

injections  for  non-  chronic  pain?  What  is  the  feasibility  of 
malignant  chronic  pain  delivering  this  procedure  to  patients  in 
regional  communities? 


Related  HTAU 
products 


Overview  of  the  regulation 
of  acupuncture  in  Alberta 
May  2001  - Information 
Paper 


One  of  two  reports  on  the 
suicide  prevention  project 


QwikNote  - February  2002 
TechNote  - May  1998 


Related  to  the  HTA  reports 
on  chronic  pain 
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2003 


Table  4:  Specification  of  HTA  products  (cont’d) 


■1 


09  Treatment  of  thoracic 
insufficiency  syndrome 
with  the  prosthetic 
titanium  rib 


Question/issue 


TechNotes  (cont’d) 


Safety  and  efficacy/effectiveness  of  using  a 
vertical  expandable  prosthetic  rib  implant  to 
treat  patients  with  thoracic  insufficiency 
syndrome.  Is  the  procedure  considered 
experimental?  What  patient  conditions  are 
eligible  for  this  treatment  and  how  many 
surgeries/hospitalizations  are  needed  over  the 
course  of  the  treatment? 


Related  HTAU 

: : ■ 


10  Minimally  invasive  hip 
arthroplasty 


11  Emergency 

department  fast-track 
system 


12  Stereotactic 
radiosurgery:  an 
update 

13  Hyperbaric  oxygen 
therapy  - recent 
findings 


14  Effectiveness  of 
computerized 
physician  order  entry 
systems 

15  Prolotherapy 


HTA  Initiatives 

Proposed  introduction  of  a new  surgical 
technique  into  CHR;  interest  in  data  on  safety, 
efficacy,  cost 


What  does  the  literature  say  about  the 
effectiveness  of  emergency  department  fast- 
track  systems? 

Information  Papers 

Current  evidence  on  the  effectiveness  of 
radiosurgery  issues  on  out  of  province 
referral,  other  indications  for  radiosurgery 


Request  from  the  College  of  Physicians  and 
Surgeons  in  relation  to  approvals  given  for 
treatments  by  the  private  hyperbaric  oxygen 
therapy  service  in  Calgary.  The  College 
wanted  updated  advice  to  take  account  of 
recent  studies  reported  in  the  literature. 

QwikNotes 

Is  there  any  research  evidence  to  show  that 
computerized  physician  order  entry  systems 
reduce  medication  errors? 


Proposed  first  stage  in  a 
more  extensive 
assessment  to  be 
coordinated  by  Calgary 
Health  Technology 
Implementation  Unit 


Two  previous  HTA 
reports,  one  TechNote 
and  two  Techscans 
(briefs),  all  used  by  AHW 

Previous  HTA  on 
hyperbaric  oxygen 
therapy  in  relation  to 
facilities  in  Alberta 


The  current  status  of  prolotherapy 


TechNote  March  1996 
Update  [brief]  May  1998 
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Table  4:  Specification  of  HTA  products  (cont’d) 


Product  Question/issue  Related  HTAU 

products 


Customized  Responses 


16  Physiotherapy/ 
physical  therapy 


Is  there  any  evidence  of  effectiveness  of 
community  physical  therapy/physiotherapy? 


17  Chiropractic  services 


Is  there  any  evidence  of  effectiveness  of 
chiropractic  services? 


18  Podiatry  services 

19  Optometric  services 


What  is  the  evidence  regarding  the 
effectiveness  of  podiatry? 

Is  there  any  evidence  of  effectiveness  of 
optometric  services? 


Expected  timelines 

Expected  timelines  for  the  various  products  are  shown  in  Table  5.  The  four  HTA 
reports  were  each  expected  to  take  a year,  as  might  be  expected  for  reports  that 
incorporate  major  reviews.  The  timelines  for  the  TechNotes  varied  from  one  to  three 
months,  continuing  the  pattern  for  this  type  of  product,  though  the  initial  timeline  for 
the  cryosurgery  publication  was  less  than  a month.  As  has  been  discussed  elsewhere  4, 
providing  considered  responses  over  such  short  time  frames  puts  pressure  on  the 
evaluation  process.  Requestors  of  TechNotes  should  provide  very  good  justification  for 
expecting  urgent  responses.  Both  QwikNotes,  as  expected,  had  very  short  timelines. 

The  first  HTA  Initiative  product  was  given  a long  timeline  because  of  the  expected 
collaboration  with  the  Calgary  HTA  program  and  the  need  to  negotiate  details. 
Timelines  for  the  second  HTA  Initiative  refers  to  editing,  review,  and  formatting  of  a 
report  that  had  already  been  prepared  outside  the  Unit.  A timeline  of  six  months  for 
the  first  Information  Paper  was  based  on  expected  time  needed  to  update  previous 
HTA  reports  on  the  topic  in  question.  The  similar  timeline  for  the  second  paper 
reflected  the  comparatively  non-urgent  nature  of  the  request. 

The  reviews  undertaken  as  customized  responses  all  had  a timeline  of  two  months. 

This  was  extremely  demanding  on  the  Unit's  work  program  and  resources. 

Acceptability  of  very  short  timelines  for  urgent  requests  needs  to  be  kept  under  review 
so  that  the  Unit  does  not  become  exposed  to  unreasonable  demands.  On  the  other 
hand,  the  Unit  has  established  an  impressive  record  over  the  years  in  handling  urgent 
requests,  and  such  activity  has  contributed  substantially  to  its  reputation. 
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Type  of  technology/topic 

A wide  range  of  topics  was  addressed,  both  in  terms  of  technologies  and  diseases/ 
conditions,  continuing  the  pattern  of  a broadly-based  HTA  agenda,  though  with  a focus 
on  non-pharmaceutical  technologies. 

Table  5:  Expected  timelines  for  assessment  reports 

Product 


Ex| 
tin 

Health  Technology  Assessments 

01  Acupuncture:  evidence  from  systematic  reviews  and  meta-analyses 

02  Prevalence  of  chronic  pain:  an  overview 


Length, 

months 


03  Multidisciplinary  pain  programs  for  chronic  pain:  evidence  from 
systematic  reviews. 

04  Suicide  prevention  strategies:  evidence  from  systematic  reviews. 

TechNotes 

; 

05  Cryosurgery  for  prostate  cancer 

06  Intracoronary  brachytherapy 

07  Osteogenic  protein-1  for  fracture  healing 

08  Trigger  Point  Injections  for  non-malignant  chronic  pain 

09  Treatment  of  thoracic  insufficiency  syndrome  with  the  prosthetic 
titanium  rib 


10/00-10/01 
5/01  - 5/02 
5/01  - 5/02 


2/01  - 2/02 

1/02-2/02, 
extended  to  4/02 

4/02  - 5/02 

10/02-12/02 

10/02-12/02 

10/02-12/02 


Hi 


HTA  Initiatives 


10  Minimally  invasive  hip  arthroplasty  6/02  - late  02 

1 1 Emergency  department  fast-track  system  01/03  - 4/03 

'■  :> : ; ■ . ; ; :# ep >: ■; p; ; . v: ;« : i ps ¥ psi p :: wp  ■; pp;  i : | ?■ . :i; : ■ • 

Information  Papers 

12  Stereotactic  radiosurgery:  an  update  10/01  - 4/02 

1 3 Hyperbaric  oxygen  therapy  - recent  findings  1 0/02  - 4/03 


12 

12 

12 

12 

1,3 

1 

2 

2 

2 


5-6 

3 

6 

6 
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Table  5:  Expected  timelines  for  assessment  reports  (cont’d) 

Product 

QwikNotes 

Expected  Length, 

timeline  months 

14  Effectiveness  of  computerized  physician  order  entry  systems 

4/02  - 4/02  <1 

15  Prolotherapy 

5/02  - 6/02  1 

Customized  Responses 

16  Physiotherapy/physical  therapy 

10/02-  1/03  2 

17  Chiropractic  services 

10/02-1/03  2 

18  Podiatry  services 

10/02-1/03  2 

19  Optometric  services 

10/02-1/03  2 
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Assessment  Products 


Allocation  to  different  series 

Decisions  to  develop  reports  in  the  HTA,  TechNote,  and  QwikNote  series  were 
consistent  with  the  expected  complexity  of  the  longer  assessments  and  the  perceived 
urgency  of  advice  for  requestors  of  non-refereed  reports.  Two  reports  were  published 
in  the  HTA  Initiatives  series  because  of  a wish  to  raise  the  profile  of  HTA  in  the 
province  and  to  document  particular  collaborative  arrangements.  One  of  the 
Information  Papers  was  a lengthy  assessment  report,  following  two  previous  HTA 
reports  on  the  same  topic.  The  other  Information  Paper  was  a brief  review  of  findings 
from  recent  HTA  reports  and  reviews.  The  Customized  Responses  were  made  available 
on  the  AHFMR  web  site  as  a non-standard  group  of  reviews. 

Some  thought  needs  to  be  given  to  the  allocation  of  reports  to  the  particular  series.  The 
products  for  2002/2003  were  spread  over  four  printed  series,  for  various  reasons.  This 
might  be  confusing  for  users  of  HTAU  products  and  unhelpful  for  dissemination. 

Issues  addressed 

Issues  addressed  in  HTAU  products  are  indicated  in  Table  6.  As  in  previous  years, 
safety,  efficacy,  and  effectiveness  were  the  most  common  topic  areas.  Eight  products 
made  some  reference  to  economic  issues,  though  there  were  none  that  included 
economic  analysis.  Four  considered  access  issues;  four  Customized  responses  referred 
to  practice  guidelines,  and  one  HTA  report  was  concerned  with  prevalence  rates. 

Unlike  some  assessments  in  previous  years,  there  was  no  consideration  of  social  or 
ethical  issues,  though  the  HTA  report  on  suicide  prevention  programs  made  some 
reference  to  social  aspects. 

Issues  addressed  in  HTA  products  will  of  course  depend  on  the  questions  that  are  being 
asked.  However,  with  some  of  the  topics  considered  in  the  2002/2003  products  there 
might  have  been  an  opportunity  to  broaden  their  scope  by  incorporating  commentary 
and  perspective  on  social  matters. 
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Table  6:  Issues  addressed  in  HTA  products 
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Collaborators  and  contractors 

The  HTA  report  on  acupuncture  was  co-authored  by  a contractor  who  provided 
important  support  in  bringing  this  project  to  a conclusion.  The  products  on  chronic 
pain  were  supported  by  input  from  an  information  sharing  group  that  included 
representation  from  AHW,  CHR,  and  Capital  Health  Region.  With  the  suicide 
prevention  program  assessment,  contact  with  Alberta  Mental  Health  Board 
representatives  was  related  to  formulation  of  the  HTA  question  rather  than  to 
collaboration.  The  HTA  Initiatives  report  on  the  emergency  department  fast-track 
system  was  written  by  a physician  from  the  University  of  Alberta  who  had  been  a 
participant  in  the  Technology  Assessment  for  Health  Care  course  offered  by  the 
Department  of  Public  Health  Sciences. 

With  the  Customized  Responses  products,  contracts  were  made  with  the  College  of 
Physical  Therapists  of  Alberta,  the  Alberta  College  of  Optometrists,  individuals  from 
the  Associated  Foot  Clinic,  and  the  College  of  Chiropractors  of  Alberta.  The  contractors 
were  used  to  review  the  selected  abstracts.  This  was  a useful  approach  to  obtain 
expertise  in  the  subject  areas  and  to  assist  the  HTAU  with  its  workload. 

A librarian  from  the  University  of  Calgary  provided  IS  support  for  the  project  on 
minimally  invasive  hip  arthroplasty  because  of  pressure  on  existing  AHFMR 
arrangements  and  the  collaboration  with  the  Calgary  HTA  program.  All  reports  in  the 
HTA  Initiatives  series  were  formatted  and  edited  by  an  external  contractor,  maintaining 
continuity  in  the  production  of  this  type  of  publication. 

Approaches  taken 

Systematic  reviews  were  undertaken  for  all  the  Health  Technology  Assessment  reports. 
One  of  these  also  had  a narrative  review  component  and  in  another  there  was  analysis 
of  administrative  and  survey  data.  Systematic  review  with  narrative  commentary  on 
cost  data  was  used  for  the  Information  Paper  on  radiosurgery. 

Two  of  the  TechNotes  were  said  to  be  based  on  systematic  review,  two  on  narrative 
review,  and  one  on  a mix  of  both  approaches.  Narrative  review  was  used  for  the  two 
HTA  Initiative  products  and  the  second  Information  Paper. 

The  approaches  taken  appear  to  have  been  appropriate  and  there  are  no  obvious  areas 
of  concern.  The  distinction  between  systematic  and  narrative  review  may  not  always  be 
entirely  clear  - even  with  the  narrative  reviews  used  for  some  products  there  would 
have  been  careful  consideration  of  selection  criteria  and  appraisal  of  quality.  A point 
that  deserves  continuing  attention  is  the  depth  of  review  and  analysis  needed  to 
appropriately  answer  the  question  that  an  HTA  is  addressing. 
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With  the  Customized  Responses,  the  approach  was  to  review  selected  abstracts  of 
systematic  reviews  on  the  different  topics.  This  was  a pragmatic  approach  adopted  to 
meet  the  time  and  scope  pressures  associated  with  these  products.  Abstract  review  was 
also  used  for  the  two  QwikNotes. 

There  was  external  review  of  all  HTA  series  reports,  one  Information  Paper,  the  HTA 
Initiatives  report  on  hip  arthroplasty,  and  two  TechNotes.  With  the  four  Customized 
Responses  there  was  external  review  of  the  articles  selected.  The  number  of  reviewers 
used  varied  from  product  to  product.  Detailed  internal  review  by  HTAU  staff  was 
undertaken  for  all  products. 

Timelines 

Table  7 gives  details  of  actual  timelines  for  the  assessment  products.  Targets  were  met 
for  all  the  urgently  required  products  (TechNotes,  QwikNotes,  and  Customized 
Responses).  This  was  an  impressive  level  of  performance  by  the  HTAU. 

All  of  the  HTA  series  reports  took  longer  than  expected.  With  the  two  on  chronic  pain, 
delay  was  in  part  associated  with  the  necessity  to  interact  with  an  external  group  and 
with  time  taken  to  define  some  aspects  of  the  project.  The  project  on  acupuncture  was 
used  as  a Teaming  experience'  for  a person  on  a short-term  placement  with  the  HTAU, 
and  there  were  overheads  associated  with  necessary  mentoring.  Protracted 
negotiations  with  the  client  were  needed  with  the  project  on  suicide  prevention 
strategies. 

It  is  not  uncommon  for  more  complex  HTA  projects  to  run  over  time;  management  of 
factors  contributing  to  delay  will  be  an  ongoing  task.  Provision  of  interim  advice  to 
clients,  as  was  undertaken  for  the  products  referred  to  here,  may  be  helpful.  However, 
an  area  that  may  need  attention  is  the  time  taken  for  review  and  publication  of  longer 
reports,  which  ran  to  five  months  for  three  of  the  HTA  products. 

The  remaining  products  were  completed  as  expected,  other  than  a small  overrun  caused 
by  delay  in  starting  the  Information  Paper  on  radiosurgery. 
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Table  7:  Eventual  timelines  for  HTA  products 
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Table  7:  Eventual  timelines  for  HTA  products  (cont’d) 
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Conclusions  reached  in  HTA  products 

Table  8 lists  brief  summaries  of  the  conclusions  reached  in  the  various  HTA  products  as 
well  as  the  questions  or  issues  that  they  addressed.  In  most  cases,  the  conclusions 
reached  appeared  to  match  the  questions  asked,  though  occasionally  some  points  were 
not  covered  or  additional  issues  were  mentioned. 


Table  8:  Conclusions  reached  in  HTA  products 


Product 


Question/Issue 


Conclusions 


“In  a word” 


01  Acupuncture: 
evidence  from 
systematic 
reviews  and 
meta-analyses 


‘For  which  conditions 
acupuncture  has 
scientific  support’ 
(effectiveness,  for 
whom,  by  whom). 


Acupuncture  appears  effective  for  Conditional 

dental  and  temporomandibular  support,  some 

pain  and  antiemesis.  For  all  other  applications. 

indications  it  is  no  more  effective 

than  the  standard  of  care  or  the 

evidence  is  too  weak  to  draw 

conclusions. 


02  Prevalence  of 
chronic  pain:  an 
overview 


Best  available 
evidence  on 
prevalence  estimates 
for  pain  not  related  to 
cancer. 


Reported  chronic  pain  prevalence  Limited  useful 
estimates  vary  widely  (10.1%  to  evidence,  need 
55.2%)  and  severe  chronic  pain  local  data, 
prevalence  is  8%  in  children  and 
11%  in  adults.  Prospective 
epidemiological  studies  should  be 
conducted  to  estimate  the  chronic 
pain  prevalence  in  Alberta. 


03  Multidisciplinary 
pain  programs  for 
chronic  pain: 
evidence  from 
systematic 
reviews 


Evidence  on  the 
efficacy/effectiveness/ 
efficiency  of 
multidisciplinary  pain 
programs  for  non- 
malignant  chronic  pain 
and  how  do  we 
establish  provincial 
needs  for  a 
multidiscipline  pain 
program? 

f 

1 


Unclear  which  treatment  (within 
the  multidisciplinary  pain 
programs)  is  responsible  for  the 
observed  improvements  in  chronic 
pain  patients  or  which  kind  of 
patients  do  best  under  a particular 
individualized  treatment  plan. 
RHAs  providing  management  of 
chronic  pain  must  establish 
appropriate  data  collection 
systems  and  assess  both  clinical 
effectiveness  of  intervention 
strategies  and  their  efficiency. 


Effectiveness 
not  established, 
local  studies 
needed. 


04  Suicide 
prevention 
strategies: 
evidence  from 
systematic 
reviews. 


What  suicide 
prevention  strategy  is 
effective? 


No  single  strategy  is  effective  in 
reducing  suicide  rate.  The 
strongest  evidence  indicated  that 
some  prevention  strategies, 
including  school-based  suicide 
prevention  programs  for  high  risk 
students  and  psychological/ 
pharmacological  treatment  for 
suicide  attempters,  appear 
promising. 


Limited 

effectiveness  for 
some  strategies, 
efficacy  not 
established  for 
others. 
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Table  8:  Conclusions  reached  in  HTA  products  (cont’d) 


05  Cryosurgery  for 
prostate  cancer 


06  Intracoronary 
brachytherapy 


07  Osteogenic 
protein-1  for 
fracture  healing 


08  Trigger  point 
injections  for 
non-malignant 
chronic  pain 


09  T reatment  of 
thoracic 
insufficiency 
syndrome  with 
the  prosthetic 
titanium  rib 


Question/issue 


What  is  the  status  of 
cryosurgery  for  the 
treatment  of  prostate 
cancer? 


The  current  status  of 
the  use  of 
intracoronary 
brachtherapy  for 
patients  with  in-stent 
restenosis  after  PTCA. 


Regulatory  status  in 
Canada  and  the  US  for 
patient  groups;  current 
practice  in  Canada; 
and  evidence 
supporting  its  use  and 
on  adverse  effects. 


The  efficacy/effectiveness  of 
cryosurgery  in  the  management  of 
prostate  cancer  is  still  not 
established. 

Evidence  from  six  RCTs  suggests 
that  the  technology  is  feasible  and 
safe  as  an  adjunct  to  treatment  of 
in-stent  restenosis,  and  reduces 
short  term  restenosis  rates. 
However,  late  thrombosis  may 
minimize  the  long-term  benefits. 

Evidence  from  one  RCT 
suggested  osteogenic  protein-1 
was  safe  and  success  rates  when 
used  in  conjunction  with 
intramedullary  rod  fixation  were 
comparable  with  those  achieved 
with  autograft  for  the  treatment  of 
tibial  non-unions.  It  is  indicated  for 
use  in  recalcitrant  long  bone  non 
unions  where  autograft  is 
unfeasible  and  alternative 
treatment  has  failed. 


Is  trigger  point  injection 
effective  for  treating 
chronic  pain?  What  is 
the  feasibility  of 
delivering  this 
procedure  to  patients 
in  regional 
communities? 


No  definitive  proof  that  trigger 
point  injection  is  more  effective 
than  acupuncture  or  placebo.  Not 
known  whether  a lack  of  multi- 
disciplinary pain  management 
programs  in  regional  areas  limits 
or  encourages  its  use,  or  what 
type  of  provider  would  achieve 
best  results. 


Safety  and  efficacy/ 
effectiveness  to  treat 
patients  with  thoracic 
insufficiency 
syndrome;  whether 
procedure  is 
experimental;  patient 
conditions  eligible  for 
this  treatment;  number 
of  surgeries/ 
hospitalizations 
needed  during  the 
treatment. 


No  major  safety  issues  were 

I identified;  early  efficacy  outcomes 
suggest  that  it  may  be  a viable 
treatment  option  for  children  with 
thoracic  insufficiency.  However, 
the  procedure  is  costly  and 
requires  frequent  hospital  visits  for 
extension  and  eventual 
replacement  of  the  device.  Its 
effect  on  pain  and  quality  of  life  is 
unknown. 


Efficacy  not 
established. 


Short  term 
safety  and 
efficacy 
established, 
long  term 
efficacy  unclear. 

Appears  to  be 
safe  and 
effective 
technology. 


; 


Efficacy  not 
established. 


Emerging 
promising 
technology, 
efficacy  not 
established. 
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Table  8:  Conclusions  reached  in  HTA  products  (cont’d) 


Product 


Question/Issue 


Conclusions 


10  Minimally 
invasive  hip 
arthroplasty 


Proposed 

introduction  of  a new 
surgical  technique; 
interest  in  data  on 
safety,  efficacy, 
cost. 


Offers  the  promise  of  lower 
morbidity,  shorter  hospital  stay, 
cost  savings,  faster  recovery. 
However,  further  studies  and 
experience  will  be  necessary  to 
establish  its  place  in  routine  care. 


11  Emergency 
department  fast- 
track  system 


12  Stereotactic 
radiosurgery:  an 
update 


Effectiveness  of 
emergency 
department  fast- 
track  system. 


Current  evidence  on 
the  effectiveness  of 
radiosurgery,  taking 
into  consideration 
out  of  province 
referral  and  newer 
indications  for  SRS. 

I 

| 


1 


13  Hyperbaric 

oxygen  therapy  - 
recent  findings 


Updated  advice  on 
effectiveness  of 
hyperbaric  oxygen 
therapy  in  different 
applications,  to  take 
account  of  recent 
studies  reported  in 
the  literature. 


Emergency  department  fast-track 
system,  as  an  adjunct  to 
emergency  departments,  appears 
to  be  efficient,  cost-effective,  safe 
and  satisfactory  to  patients.  It 
should  incorporate  timely  and 
accurate  reporting  of  visits  to 
patients’  primary  care  physicians. 

SRS  should  continue  to  be  made 
available  to  appropriately  selected 
patients  with  metastatic  disease, 
arteriovenous  malformations  and 
acoustic  neuroma;  it  should  not 
be  offered  for  treatment  of 
Parkinson’s  disease,  trigeminal 
neuralgia  or  epilepsy.  Any 
referral  of  patients  for  SRS  should 
be  to  centres  of  excellence,  taking 
account  of  other  appropriate 
treatment  options. 

There  is  support  for  use  of 
hyperbaric  oxygen  therapy  for 
decompression  sickness,  gas 
embolism,  gas  gangrene,  carbon 
monoxide  poisoning, 
osteoradionecrosis,  diabetic 
wounds,  and  necrotising  soft 
tissue  infections.  There  is  no 
consensus  on  support  for  its  use 
in  osteomyelitis,  thermal  burns, 
soft  tissue  radionecrosis, 
compromised  skin  grafts,  and 
many  other  conditions. 


“In  a word” 

Emerging 
promising 
technology,  not 
yet  established. 


Effective 

technology 


Conditional 
support,  some 
applications. 


Conditional 
support,  some 
applications. 
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Table  8:  Conclusions  reached  in  HTA  products  (cont’d) 


Question/Issue 


14  Effectiveness  of 
computerized 
physician  order 
entry  systems 
(CPOE) 

Is  there  any 
research  evidence 
to  show  that 
computerized 
physician  order 
entry  systems 
reduce  medication 
errors? 

Evidence  from  three  RCTs 
indicated  that  medical  errors 
decreased  after  implementation  of 
CPOE  system.  CPOE  systems 
are  likely  most  effective  when  part 
of  an  electronic  medical  record 
system  allowing  full  access  to  a 
patient’s  clinical  information. 

Effective 

technology. 

15  Prolotherapy 

The  current  status  of 
prolotherapy. 

Results  of  abstracts  of  three 
RCTs  noted  - no  conclusions 
provided. 

Ineffective 

technology. 

16  Physiotherapy/ 
physical  therapy 

17  Chiropractic 
services 

18  Podiatry  services 

19  Optometric  services 

Is  there  any 
evidence  of 
effectiveness  of 
these  services? 

No  overall  conclusions, 
documents  summarized  findings/ 
conclusions  in  reviewed 
abstracts.  However,  available 
data  indicate  that  there  is  such 
evidence. 

;| ’ 1 ; ; 

Widely  used 
health  services, 
essential  in 
numerous 
applications. 
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Dissemination 


Brief  details  of  approaches  taken  to  dissemination  of  HTA  products  are  given  in  Table  9. 
Paper  copies  of  all  products  were  sent  to  the  primary  targets  and  electronic  copies  of  all 
but  three  were  e-mailed.  There  seemed  to  be  direct  discussion  with  the  primary  target 
in  relatively  few  cases.  Discussion  was  ongoing  for  the  chronic  pain  products  and  the 
suicide  prevention  programs  report,  providing  some  offset  against  the  long  time 
needed  to  complete  the  publications.  A presentation  was  made  on  one  of  the 
QwikNotes  and  there  was  some  discussion  with  AHW  regarding  the  Information  Paper 
on  radiosurgery.  Potentially  this  is  an  area  that  could  be  improved,  with  more  active 
follow-up,  though  it  is  appreciated  that  this  is  not  always  easy  to  arrange  and  that  it 
places  further  demands  on  resources. 

Wider  dissemination  was  carried  out  via  the  standard  HTAU  circulation  list,  the 
AHFMR  web  site,  the  HTA  database,  and  the  INAHTA  web  site  in  many  cases.  There 
were  also  several  conference  presentations  and  in  some  cases  secondary  circulation  of 
products  by  other  organizations.  The  invitation  to  publish  the  chronic  pain  prevalence 
report  in  the  International  Association  for  the  Study  of  Pain  Clinical  Updates  was  useful 
for  the  dissemination  process.  Requests  for  publications  and  citations  in  other  agencies' 
reports  give  an  indication  of  visibility  (and  potential  secondary  impact). 

In  just  one  case  - the  report  on  the  emergency  department  fast-track  system  - more 
elaborate  short-term  dissemination  arrangements  were  made,  including  involvement  of 
the  AHFMR  Communications,  a news  release  to  print  and  electronic  media,  and  a 
media  interview.  The  opportunity  for  supporting  dissemination  of  other  HTA  products 
is  an  issue  that  merits  discussion. 
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Table  9:  Approaches  to  dissemination  of  HTA  products  (cont’d) 

Product  Primary  Target  Wider  Dissemination 

Paper  E-mai!  Discussion  Person(s) 

Responsible 
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Impact  of  HTA  products 


Tables  10  to  13  give  information  related  to  the  impact  of  the  various  assessment 
products.  For  the  most  part,  the  information  reflects  the  direct  experience  of  HTAU 
staff  in  managing  individual  projects,  and  in  a number  of  cases  there  is  also  supporting 
information  through  documents  from  clients  and  other  organizations.  The  data 
therefore  give  an  important  "snapshot"  of  the  impact  of  the  HTA  program,  but  it 
should  be  borne  in  mind  that  there  is  no  detailed  study,  for  example  using  survey 
methodology,  to  substantiate  all  the  information  and  opinion  given  here. 

Areas  of  impact 

Table  10  outlines  the  areas  of  impact  of  HTA  studies,  using  the  categories  adopted  for  a 
project  on  reporting  the  impact  of  HTA  reports  that  is  currently  being  undertaken  by 
INAHTA.  Only  one  product  had  no  apparent  impact  (though  this  was  reviewed  by  an 
external  information  group).  All  other  products  were  at  least  considered  by  decision 
makers.  In  six  cases,  conclusions  or  recommendations  were  accepted.  Material  from 
five  products  was  incorporated  into  policy  documents  and  14  assessments  were  used  as 
reference  material.  One  HTA  report  was  linked  to  changes  in  practice  and  six  products 
had  other  impacts,  including  input  to  future  research  on  the  technologies  considered 
and  expectations  of  future  collaborations  with  the  HTAU. 

Level  of  impact 

Table  11  includes  opinions  of  HTAU  staff  on  the  level  of  impact  associated  with  the 
HTA  products  and  gives  some  supporting  details.  Two  TechNotes  on  cryosurgery  and 
intracoronary  brachytherapy  were  thought  to  have  had  major  impact  on  subsequent 
decisions  on  these  technologies.  A further  fifteen  products  were  considered  to  have 
provided  some  input  to  decisions.  Possibly  this  is  a conservative  view,  as  in  several 
cases  there  appeared  to  have  been  significant  developments  that  flowed  directly  from 
work  done  on  the  assessment  products.  In  one  case,  the  product  received  some 
consideration  by  a decision  maker  and  in  one  the  impact  was  minimal.  The  details  in 
the  table  give  an  indication  of  action  taken  and  use  of  the  products  by  different 
stakeholders  in  the  province. 

Indirect  impact 

Table  12  gives  information  on  indirect  impacts  of  eight  HTA  products.  With  four  of 
these  there  was  some  influence  on  other  targets  within  Alberta,  with  likely  influence  on 
future  policy  or  administrative  decisions.  Six  products  were  accessed  by  organizations 
outside  Alberta  and  provided  some  input  to  HTA  activities  undertaken  by  other 
agencies. 
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Types  of  decisions 

Table  13  gives  an  indication  of  the  types  of  decisions  that  were  informed  by  the  various 
HTA  products,  using  categories  from  the  current  INAHTA  project.  This  draws  on  all 
available  information  and  does  not  necessarily  reflect  the  exact  questions  or  issues  that 
the  HTA  products  were  required  to  address.  Coverage  decisions  were  the  most 
common,  related  to  10  and  possibly  11  products.  In  four  cases  there  were  capital 
funding  decisions,  referral  for  treatment  in  two  (possibly  three),  program  operation  in 
seven,  guideline  formulation  in  two,  influence  on  routine  practice  in  three  (possibly 
four),  and  indications  for  further  research  in  five.  This  summary  reflects  the  range  of 
issues  addressed  by  products  from  the  HTAU. 

Overall,  the  picture  that  emerges  from  this  summary  of  impacts  is  encouraging.  While 
detail  was  sometimes  limited,  it  seems  clear  that  products  from  the  HTAU  had  a useful 
impact  in  most  cases,  and  that  often  this  was  achieved  by  products  with  very  tight 
deadlines. 
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Table  11:  Level  of  impact  of  HTA  products 
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Minimal  Some  Input  to  Major 

consideration  decisions 
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Table  11:  Level  of  impact  of  HTA  products  (cont’d) 

Opinion  on  level  of  impact  Details 

Minimal  Some  Input  to  Major 
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Profile  of  an  HTA  Program 

The  AHFMR  Health  Technology  Assessment  Unit,  2002  - 2003 
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Provoked  responses  from  physicians  with  direct  t0  jts  report 
interests  in  hyperbaric  oxygen  therapy  services. 
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Resource  and  Educational  Products 


1.  The  first  of  the  three  publications  in  this  group  was  the  latest  version  of  " Health 
technology  assessment  on  the  net",  which  was  first  published  in  1999  and  has  been 
updated  annually  since  then. 

This  is  a widely  known  publication  that  has  proved  to  be  a useful  resource  for  those 
working  in  HTA.  It  has  been  made  available  through  the  AHFMR  web  site  and 
through  hard  copy  and  numerous  other  outlets.  These  include  conference 
presentations  (ISTAHC  and  the  Canadian  Health  Libraries  Association),  the  HTA 
course  at  the  University  of  Alberta,  the  National  Library  of  Medicine's  Etext  on 
Health  Technology  Assessment  Information  Resources  (links  to  the  guide),  and 
through  several  other  web  sites,  for  example,  the  NeLH  (National  Electronic  Library 
of  Health),  ScHARR,  DACEHTA  (Denmark),  VATAP  (USA),  and  the  Evidence- 
Based  Medicine  group  (Italy).  The  authors  have  been  responsible  for  dissemination. 

Requests  for  the  document  have  been  received  from  all  over  the  world.  For 
example,  NZHTA  (New  Zealand)  has  used  the  guide  in  their  training;  the  Health 
Council  of  the  Netherlands  has  recommended  the  guide  to  their  staff;  the  Singapore 
Ministry  of  Health  has  requested  copies;  and  the  Newfoundland  Ministry  of  Health 
requested  copies  for  staff  members.  In  Alberta,  the  guide  has  been  used  by 
librarians  at  the  University  of  Alberta  John  Scott  Health  Sciences  Library  and 
included  in  their  Evidence-Based  Healthcare  Information  Resources  web  page. 

This  is  a useful,  ongoing  activity  for  the  HTA  program  that  requires  only  limited 
resources  to  maintain.  Overall,  impact  would  be  hard  to  assess  but  certainly 
AHFMR  is  well  known  as  a useful  source  of  information  in  this  area. 

2.  The  first  of  the  HTA  Initiatives  products  "Local  health  technology  assessment:  a guide  for 
health  authorities"  was  developed  to  assist  in  building  HTA  capacity  and  competence 
at  the  local  level  in  Alberta.  Staff  in  RHAs  were  the  main  target.  The  request  for  the 
product  came  from  the  Director  of  HTA,  with  the  report  seen  as  being 
complementary  to  other  publications  in  this  series.  It  was  completed  in  2002  (start 
July,  review  November,  published  December). 

This  is  a guideline  covering  elements  of  HTA,  based  on  narrative  review  and 
commentary.  Dissemination  was  through  standard  circulation  procedures  plus 
discussion  by  the  author  with  staff  at  Foothills  Hospital,  Calgary  and  at  a workshop. 

Impact  of  this  product  is  hard  to  assess.  There  was  interest  in  the  document  from 
the  CHR,  with  verbal  advice  from  the  Department  of  Surgery  that  it  had  been 
helpful  in  considering  approaches  to  new  technologies.  There  has  also  been  interest 
from  individuals  in  Italy  and  the  Philippines,  via  conference  contacts.  It  is  unclear, 
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from  the  information  available,  whether  there  has  been  any  additional  impact  within 
RHAs  in  Alberta,  which  were  the  main  target  for  this  product. 

3.  The  suggestion  for  development  of  the  final  product,  " Elements  of  effectiveness  for 
health  technology  assessment  programs"  came  from  discussion  between  the  author  and 
the  Director  of  HTA,  and  followed  earlier  work  in  this  area  at  the  Department  of 
Public  Health  Sciences,  University  of  Alberta,  which  had  been  presented  at  an 
ISTAHC  meeting.  The  issue  addressed  is  the  factors  that  define  and  contribute  to 
the  effectiveness  of  HTA  programs.  It  was  seen  as  a contribution  to  building  HTA 
competence. 

The  project  was  started  in  September  2002  and  published  in  March  2003.  It  is  an 
extended  commentary  with  a focus  on  HTA  program  effectiveness.  Drafts  were 
given  to  a number  of  HTA  contacts  in  Canada  and  elsewhere  but  there  was  no 
feedback  during  preparation. 

The  primary  target  for  this  work  was  really  the  HTAU,  and  the  immediate 
consequence  is  the  present  paper.  In  addition  to  dissemination  through  standard 
circulation  methods,  the  work  has  been  presented  at  seminars  and  conferences. 
General  interest  in  the  approach  has  been  expressed  by  agencies  in  Canada,  the 
Netherlands,  and  Singapore,  but  there  is  as  yet  no  specific  indication  of  impact. 
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Little  information  on  governance  was  provided  in  the  responses  from  HTAU  staff,  and 
issues  in  this  area  can  be  dealt  with  only  to  a limited  extent  in  a project  of  this  sort.  A 
more  direct  and  detailed  approach  would  be  needed  to  explore  relevant  issues. 

With  all  HTA  products  and  activities  of  the  HTAU,  reports  are  prepared  and  submitted 
to  the  Chief  Executive  Officer  and  the  Trustees  on  a regular  basis.  Regular  updates  are 
provided  to  the  AHFMR  Health  Advisory  Committee  and  Health  Research  Advisory 
Committee.  The  committees  also  provide  advice  for  future  opportunities  and  activities. 

With  the  Customized  Responses,  information  provided  indicated  that  the  Chief 
Executive  Officer  and  Trustees  of  the  AHFMR  were  informed  of  the  initiation  and 
progress  of  the  project.  Their  feedback/ directive  was  to  proceed  with  the  project  but 
with  the  understanding  that  other  projects  would  be  delayed. 

A positive  aspect  was  that  the  need  for  transparency  and  accountability  was  clearly 
communicated  between  the  AHFMR  and  the  client.  It  was  made  clear  that  reports 
prepared  by  HTAU  would  be  released  at  some  point  in  the  future. 

The  only  other  area  related  to  governance  was  a delay  in  finalizing  a contract  associated 
with  the  project  on  radiosurgery. 
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Detailed  discussion  of  resources  and  staff  issues  are  beyond  the  scope  of  this  paper  and 
would  require  detailed  local  input  and  analysis.  However,  some  points  arise  from  the 
information  made  available. 

There  were  ten  authors  for  the  assessment  products,  six  of  whom  were  on  short-term 
placements  or  external  to  the  HTAU.  In  general,  assessors  were  involved  with  a range 
of  topics  and  most  had  a role  in  preparation  of  longer  term  products.  The  four 
permanent  assessors  in  the  HTAU  were  all  involved  in  the  preparation  of  the 
Customized  Responses,  which  would  have  required  a substantial  commitment  of  their 
time.  The  HTA  Resource  products  had  three  authors. 

IS  support  was  provided  by  the  AHFMR  Librarian  for  10  products  and  by  other  persons 
for  12.  These  statistics  reflect  the  continuing  requirements  of  the  HTA  program  and  the 
limited  IS  support  directly  available  from  the  AHFMR.  One  person  provided  other 
support,  which  included  formatting  and  editing  of  draft  reports,  for  all  the  Unit's 
products.  This  is  a situation  that  has  been  with  the  HTAU  for  some  time  and  was  an 
issue  raised  in  HTAI  #9  ("Is  there  sufficient  support  to  provide  an  efficient  HTA  program?") 

1.  A contractor  provided  formatting  and  design  services  for  four  products  in  the  HTA 
Initiatives  series,  an  arrangement  that  worked  well. 

This  summary  takes  no  account  of  the  staff  resources  used  in  internal  review  and 
technical  editing  of  HTA  products.  This  is  a demanding  area  that  forms  part  of  the 
routine  operation  of  the  HTAU  and  contributes  considerably  to  the  quality  of  its  output. 
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Table  14  brings  together  information  given  previously  on  issues  addressed,  policy 
areas,  HTA  findings,  and  opinion  on  impact,  to  give  some  synthesis  of  overall  activity. 
These  details  give  a general  picture  of  the  HTA  products  and  their  impact.  There  is  an 
overall  indication  of  questions  that  were  addressed  and  assessment  findings,  and  the 
use  made  of  these,  giving  assurance  of  appropriate  activity  for  the  HTAU  use  of 
assessment  to  inform  policy  decisions. 
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Discussion 


This  paper  brings  together  many  details  of  the  performance  of  the  HTAU  at  the 
AHFMR  as  reflected  in  the  products  that  were  completed  from  April  2002  to  March 
2003,  though  it  is  only  a partial  overview.  The  data  and  commentary  are  based  on  the 
information  provided  by  HTAU  staff,  the  publication  record,  and  impressions  obtained 
by  the  author  through  periodic  contact  with  the  Unit.  Further  information,  for  example 
on  resources,  would  be  needed  to  give  a comprehensive  profile  and  would  require 
additional  input  from  within  the  AHFMR.  There  has  been  no  detailed  review  of  the 
individual  products.  Nor  has  there  been  any  discussion  of  other  HTAU  activities,  such 
as  presentations  at  seminars  and  conferences. 

The  available  information  shows  that  there  was  good  performance  from  the  HTAU, 
with  particular  strengths  in  some  of  the  literature  reviews  and  in  responses  to  urgent 
requests.  Output  of  normal  types  of  HTA  products  by  core  staff,  particularly  longer 
term  projects,  was  influenced  by  the  work  on  the  Customized  Responses. 

In  HTAI  #9  it  was  suggested  that  important  elements  for  an  HTA  program,  which 
contribute  to  its  continuity,  are  Need  for  HTA,  Governance,  Efficiency,  Quality, 
Coverage,  and  Impact  (see  Figure  2). 

Figure  2:  Elements  of  Effectiveness 
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The  information  available  on  the  HTAU  for  its  2002/2003  products  is  encouraging  on 
these  elements: 

Need  for  HTA:  The  services  of  the  HTAU  clearly  continue  to  be  in  demand  from 
decision  makers  in  Alberta  health  care.  The  Customized  Responses  provided  a 
diversion  from  expected  HTA  activities. 

Governance:  Values  appear  to  have  been  upheld.  The  nature  and  degree  of  interaction 
of  governance  with  the  HTA  program  is  not  clear  from  the  information  available. 
Continued  support  to  ensure  that  requests  from  government  are  reasonable  and 
realistic  1 appears  essential  in  view  of  pressures  on  the  HTAU  that  arose  during  this 
period. 

Efficiency:  Efficiency  in  terms  of  handling  short-term  requests  was  high.  Efficiency 
with  longer  term  activities  is  less  easy  to  assess,  in  part  because  of  the  involvement  of 
other  agencies.  Overall  output  was  a product  every  2.4  weeks,  an  HTA  product  every  3 
weeks,  and  a longer  HTA  product  every  7.4  weeks.  These  rates  seem  very  acceptable. 

Quality:  Quality  of  products  was  not  considered  in  any  detail  for  this  paper.  Quality  of 
reviews  appeared  to  be  very  good.  Ongoing  consideration  on  the  relationship  of  HTA 
product  content  to  formulation  of  HTA  questions  would  be  useful. 

Coverage:  A broad  range  of  topics  was  addressed  in  the  Unit's  products,  continuing  the 
pattern  from  previous  years.  The  balance  between  follow-up  assessments  and  new 
topics  will  continue  to  be  an  issue  to  monitor. 

Impact:  Many  products  had  a useful  impact  on  decision  making. 

Some  areas  that  deserve  consideration  in  future  management  of  the  HTAU  are  listed 
below.  Several  of  them  are  issues  that  have  been  around  for  some  time  and  may  need 
ongoing  management  rather  than  definitive  resolution. 

• Balance  of  long-term  and  short-term  projects.  Most  of  the  products  for  2002/2003  were 
either  long-term  assessments  or  responses  to  urgent  requests.  There  was  not  much 
middle  ground. 

• Allocation  to  series.  HTA  products  were  spread  over  four  printed  series,  for  various 
reasons.  This  might  be  confusing  for  users  of  HTAU  products. 

• Acceptability  of  very  short  timelines  for  "urgent"  requests  needs  to  be  kept  under 
review  so  that  the  Unit  does  not  become  exposed  to  unreasonable  demands. 

• The  time  taken  for  review  and  printing  of  longer  reports  should  be  monitored. 

• The  levels  of  analysis  needed  in  assessment  products  to  appropriately  address  the 
questions  or  issues  raised  should  be  kept  under  review. 
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• There  may  be  an  opportunity  to  usefully  broaden  the  scope  of  assessment  activities, 
for  example  through  consideration  of  social  issues. 

• The  resource  products  are  a useful  component  of  the  Unit's  output,  but  it  is  hard  to 
judge  their  overall  impact. 

• The  limited  support  from  within  the  AHFMR  for  dissemination  activities  is  an  issue 
that  requires  discussion. 

• Support  staff  resources  for  the  HTAU  are  modest.  This  appears  to  be  an  area  of 
vulnerability. 
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Appendices 


Appendix  A:  Publication  product  series  produced  by  the 

HTAU 

Health  Technology  Assessments:  Comprehensive  appraisals  of  health  technologies; 
providing  a synthesis  of  data  from  the  literature  or  reporting  on  empirical  studies. 
Subject  to  external  review. 

TechNotes:  Brief  responses  to  requests  for  rapid  advice,  with  limited  analysis.  These 
are  not  subject  to  external  review. 

Information  Papers:  Publications  providing  information  on  health  technology  topics. 
No  assessments. 

Health  Technology  Assessment  Initiatives:  Publications  with  the  primary  aim  of 
introducing  and  integrating  an  applied  HTA  process  into  a RHA's  program  operations. 

QwikNotes:  Very  rapid  responses  to  urgent  requests,  with  limited  assessment, 
provided  electronically. 
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Appendix  B:  Information  Collection  Form 

HTAU  profile,  2003 


Title: 

AHFMR  contact: 

Origin  of  request: 

FORMULATION  OF  THE  QUESTION 

Date  of  request: 

What  was  the  question/issue? 

Expected  timeline: 

Was  there  discussion  to 
confirm  issue/approach? 

Any  relationship  to  other 
reports  (AHFMR  or  other?) 

HTA  PRODUCT 

Type  (series) 

Reason  for  this  type  of  report? 

Authors: 

Any  external  collaboration? 

IS  support:  Other  support: 

(indicate  who,  why,  role) 

Was  collaboration  through  contract,  other  agency,  free  advice/ 
input,  other? 


Start  date: 

Review  date:  Publication  date: 

Approach  [tick  one  or  more]: 

- systematic  review 

- narrative  review 

- admin,  data  analysis 

- cost/economic  analysis 

Issues  addressed  [tick  one  or  Reviewers: 
more]: 

- safety 

- efficacy 

- effectiveness 

- survey 

- economic  impact 

- other 

- access 

- social 

- ethical 

- other 
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Report  title 

DISSEMINATION 

Approach  for  primary  target:  Mail  paper  copy 


Approaches  for  wider 
dissemination 

E-mail 

Face  to  face  discussion 
Longer  dialog 
Other 

Standard  circulation  list, 
newsletter,  web  site 

HTA  database 

INAHTA  web  site 

Media  release/conference 

Meeting/seminar 

Conference  presentation 

Other 

Persons  involved  in 
dissemination 

Author(s)  (Details) 

Other  HTAU 
Other  AHFMR 
External  to  AHFMR 

GOVERNANCE,  RESOURCES,  STAFF  & STRUCTURE 

Any  information  or  issues  regarding  this  HTA  related  to  Governance  (e.g.  CEO,  Board  of  Trustees, 
Minister),  Resources,  Staff  and  Structure  of  HTAU? 


Alberta  Heritage  Foundation  for  Medical  Research 
Health  Technology  Assessment 


55 
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Report  title 

General  opinion  on  impact 

(GIVE  DATE  FOR  THIS 
ENTRY) 


What  were  main  indications  of 
impact? 


Were  there  any  indications  of 
impact  on  other  organizations 
or  individuals? 

Have  there  been  any  requests 
for  follow  up  HTA  work  on  this 
topic? 


IMPACT 

1 . Minimal  Was  there  any  impact  on  the  main 

target? 

2.  Some  consideration  of 

HTA 

3.  Some  input  to  decisions 

4.  Major  influence  on 

decisions 

[Tick  one  or  more]  (Details) 

1 . No  apparent  impact 

2.  HTA  considered  by 

decision  - maker 

3.  HTA  recommendations/ 

conclusions  accepted 

4.  HTA  demonstrated  that 

technology  met  specific 
program  requirements 

5.  HTA  material  incorporated 
into  policy  or  administrative 
documents 

6.  HTA  information  used  as 

reference  material  

7.  HTA  linked  to  changes  in 

practice 

8.  Other  [please  specify] 


(Details) 
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